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Abstract 


An individual's social style is determined by behavioral patterns in the interactions with their peers. Some 
studies suggest that social style may influence the way in which an individual's performance is evaluated. We 
studied the effects that speakers' and evaluators' social styles have on the marks given for end-of-term 
presentations in a project engineering master's course. The participants completed a self-evaluation exercise 
that classified their social styles into one of four categories: Driver, expressive, analytical, or amiable. Students 
individually rated the content and appearance of their classmates' presentations. A statistical analysis of these 
scores revealed that the speaker's social style had a significant effect on the marks received for content and 
appearance. The evaluator's social style also demonstrated a statistically significant effect on the marks given 
for appearance, though not for content. Students with expressive social style received the highest scores, while 
the analytical style received the lowest scores. These results reiterate the necessity to train students as 
evaluators in order to reduce bias when evaluating their classmates and co-workers during their academic and 
professional careers. 

Keywords- Social style, rating, evaluation, presentations. 


1 INTRODUCTION 

An individual's social style defines a set of behavioral patterns in their interactions with others. The social style 
model developed in the 1950s by Dr Merril (Tracom, 1991) uses simple questionnaires in order to evaluate two 
scales of social style. These scales were named assertiveness and sensitivity. Assertiveness is defined as the 
effort a person makes to influence the thoughts and actions of others. On the other hand, responsiveness 
describes the tendency to express one's feelings. 

The quadrants that result from the combination of the two scales -assertiveness and responsiveness- define 
four social styles: Analytical, driver, expressive, and amiable (see figure 1). People who are not very assertive 
and not very sensitive are considered to be analytical. These people tend to be prudent, reflexive, and 
objective, and may also be considered cold and indecisive. People with low responsiveness and high 
assertiveness are drivers. The members of this group are decisive, independent, sincere and efficient, but also 
are sometimes perceived as brusque in their interactions with others. People with high assertiveness and high 
emotional responsiveness make up the expressive group. These people are perceived as outgoing, enthusiastic, 
persuasive and spontaneous, while are sometimes seen as impertinent and distractive. Lastly, people with low 
assertiveness and high expressiveness compose the amiable group. These people are perceived as diplomatic, 
cooperative, and patient but also permissive and dependent. 
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High responsiveness - "emotes" 


Amiable 


Low assertiveness 
"asks" 



Assertiveness 


Expressive 


High assertiveness 
"tells" 



Low responsiveness - "self-control" 


Figure 1. Social styles. Two-dimensional space defining the personality traits analyzed in this study. The "assertiveness" axis 
indicates the individual's tendency to impose their ideas or to go along with the ideas of the group. The "responsiveness" 
axis indicates the tendency to display one's emotions, as opposed to emotional self-control 


The effect of social style on peer reviews has been investigated in previous studies, in industrial (Antonioni & 
Park, 2001) and academic environments (May & Gueldenzoph, 2006). However, to our knowledge, there are no 
studies in which this effect was analyzed on peer reviewers rating short in-class presentations. The reviewed 
literature allows us to foresee distortions in the marks given and received as a result of the combinations of the 
social styles of the speaker and evaluator. Specifically, the perceived similarity theory states that the similarities 
between individuals are an influential factor in the ratings given in similar pairs. These similarities can be 
identified either by behavioral or cognitive factors. In particular, the degree of perceived similarity has been 
described as a source of bias (Mumford, 1983) and imprecision (Orpen, 1994) in peer review. 

The adaptation to the new European Higher Education Area (EHEA) requires the university professor to apply 
teaching methodologies in which the students are the protagonists of their own education. These 
methodologies introduce evaluation systems which allow for a more complete following of the individual's 
learning process, facilitating their education and tutoring. 

In addition to the main topics of the course, generic competencies of each degree program are also included. 
The level at which the skills have been learnt is also assessed (Scriven, 1967) (Hall & Burke, 2003) (Kaftan, Buck 
& Haack, 2006). All of the study plans at the School of Industrial Engineering of Barcelona (ETSEIB) include the 
seven generic competencies specified in the framework of the Universitat Politecnica de Catalunya, 
BarcelonaTech (UPC): Determined usage of information resources, sustainability and social commitment, 
efficient oral and written communication, self-learning, innovation and entrepreneurship, a third language 
(English), and teamwork. 

The "project engineering" course in Industrial Organization masters' degree (a second cycle engineering 
program planned to extinction with the incoming EHEA plan), has used active teaching methodologies since the 
year 2000: Problem and project-based education, cooperative learning, and group e-portfolios. 

In this course, students are also trained on effective communications skill. The end-of-course evaluation 
specifically challenges on oral communication skills (Linguistic Services, 2012) with the aid of a poster 
(hereafter: Presentations). 

Historically, in this course, the Presentations were not rated by peer reviews, although students were 
encouraged to orally critique the presentations at will. During the planning of the 2011-2012 academic year, the 
professors of the Projects course considered the possible benefits of structuring a peer review system for the 
Presentations; In particular, those for the competence of effective oral communication. In order to assess their 
peers, each student will have to use their knowledge and acquired abilities in a critical manner (Marin Garcia, 
2009). 

A simple scoring rubric was elaborated to guide peer reviews of the Presentations (see Figure 2). Although a 
rubric is used to guide the evaluation, we must consider how the social style of each student could influence 
the evaluation of their peers. We decided to carry out a study to determine whether a student's social style can 
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bias the Presentation peer review. In the study's initial considerations, the social styles of both the evaluator 
and the speaker being evaluated were identified as possible sources of bias. 

In this article we present an exploratory study aimed to determine the influence of social style in the act of 
evaluating engineering project presentations in the context of higher education. 


Group evaluated: 

Evaluator: Evaluator group: 


0-10 

Content points for the presentation max) 

The current problem is clear and realistic 


Situation quality evaluation criteria 


Internal, external, and third party users are identified 


The desires of different users is presented 


Users affected by the project are identified 


Ergonomic design for the operators 


Interrelations with external systems developed 


Design for security 


Design for reliability 


Alternatives presented 


Description of the final solution 


Economic viability of the solution 


Technical viability of the solution 


Viability and environmental sustainability of the 
solution 


0-10 

Appearance evaluation points for the presentation max ) 

Adequate poster distribution 


Legibility of the poster 


Attractive composition 


Titles, author names, size according to norms 


Presentation under 20 minutes 


All members have relevant roles 


Cohesion between each member's intervention 


Group: usage of technical language 


Group: adequate pace 



Rate each speaker (0 to 10) 

The same number cannot be repeated for 
more than one speaker 

Speaker 

Name 

Content 

Appearance 

1 




2 




3 




4 




5 




6 




7 




8 





Figure 2. Rubric used for peer reviews 


2 METHODS 

Six groups of six to eight students agreed to participate in the study during the end-of-course project 
presentations. During these presentations, all students presented parts of their projects in two to four minute 
speeches, followed by questions from a panel of reviewers. The rest of the class evaluated each student's 
presentation, giving scores for content quality (content, hereafter) and performance quality during the 
presentation (appearance, hereafter). The marks were given on a 0 to 10 point scale. In order to encourage the 
evaluation process, the evaluators were instructed to assign integer values without score repetitions among the 
members of each group. Data were collected during the Presentation sessions. 

A total of 32 students (26 male) participated in the study. The group carried out a total of 728 peer reviews. In 
the statistical analyses, the classification of the social style of each student was used as the independent 
variable, assessing its effect on the evaluations given and received for presentation content and appearance. 

Before the presentations, the social style of each student was determined by means of a self-administered 
questionnaire on social style, distributed by Wilson Learning (online). This questionnaire consists of self- 
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descriptive responses regarding 20 items. The questionnaire is completed and corrected in less than 20 
minutes. The completion of the questionnaire supported a practical exercise on the social style concepts. 

The same statistical analysis of the data was carried out for the ratings given for content and appearance in 
three phases: First, four factorial variance analyses (ANOVA) were used to separately describe the effect that 
the speakers' and evaluators' social styles have on evaluation. Second, a two-way ANOVA was used at four 
levels (social styles of the evaluator and the speaker) to break down the marginal effects of the social styles. 
Lastly, a post-hoc analysis was carried out on the average marks for each pair of social styles. The ANOVA 
analyses allow, by means of lineal regressions, a split of the variance of a dependent variable (in this case, the 
marks), explained by a number of independent factors (social styles of the evaluator and speaker, in the case of 
our study). The magnitude of the explained variance is compared with the residual error in order to determine 
its statistical significance (with reference to Fisher's F distribution and expressed with p, the probability of a null 
hypothesis). When an analysis incorporates multiple comparisons, the significance threshold of p is adjusted by 
means of the Bonferroni technique to compensate the effect of repeated measurements. 

3 RESULTS 

The distribution of the social styles observed in the sample group was: 44% expressive (n=14, 11 males), 22 % 
driver (n=7, 4 males), 22 % amiable (n=7, 7 males) and 12% analytical (n=4, 4 males). 

3.1 Single-factor ANOVA Analysis 

In the first analysis series (see Table 1), the comparison of content scores received in function of the speaker's 
social style revealed a significant effect from the social style factor (F=5.879; p=0.001). The post-hoc analyses 
showed that subjects of the expressive social style received higher marks than the analytical and driver styles 
with a statistical significance which exceeds the threshold p<0.05, corrected by Bonferroni. There was also a 
tendency to receive higher marks than the amiable social style, though without reaching the threshold of 
significance (p=0.091). 



Single-factor 

ANOVA 

Social style 

n 

Average 

Standard deviation 

Standard 

error 

Content mark 

received. 

Speaker style 

F=5.879 

p=0.001 

Analytical 

93 

6.59 

1.99 

0.21 

Driver 

173 

6.90 

1.89 

0.14 

Expressive 

323 

7.38 

1.81 

0.10 

Amiable 

139 

6.92 

1.82 

0.15 

Total 

728 

7.08 

1.88 

0.07 

Appearance 

mark received. 
Speaker style 

F=4.601 
p=0.003 

Analytica 

93 

6.64 

1.92 

0.20 

Driver 

173 

6.73 

2.13 

0.16 

Expressive 

323 

7.25 

1.89 

0.11 

Amiable 

139 

6.75 

1.84 

0.16 

Total 

728 

6.96 

1.96 

0.07 

Content mark 
given. 

Evaluator style. 

F=2.498 

p=0.059 

Analytical 

94 

6.82 

1.90 

0.20 

Driver 

160 

7.12 

1.96 

0.16 

Expressive 

349 

7.23 

1.83 

0.10 

Amiable 

125 

6.78 

1.85 

0.17 

Total 

728 

7.08 

1.88 

0.07 

Appearance 
mark given. 

Evaluator style. 

F=4.312 

p=0.005 

Analytical 

94 

6.53 

2.09 

0.22 

Driver 

160 

6.95 

2.15 

0.17 

Expressive 

349 

7.19 

1.80 

0.10 

Amiable 

125 

6.63 

1.96 

0.18 

Total 

728 

6.96 

1.96 

0.07 





Table 1. Results of the single-factor variance analysis. Descriptive statistics of the sample and results of the single-factor 
ANOVA tests. A total of 728 content and appearance evaluations were included in the mean score comparisons, split by the 
social style of the evaluator and the speaker. Statistically significant differences in average marks were identified between 
social styles. The * sign indicates a difference in averages with a significance of p<0.05 corrected according to Bonferroni 
(n=120). The ** sign indicates a significance of p<0.01; with the same Bonferroni correction 
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The analysis of appearance scores according to social style revealed a significant effect from the social style 
factor (F=4.601; p=0.003). Post-hoc analyses showed that the subjects of the expressive social style received 
higher marks than the analytical and driver styles with a statistical significance that exceeds the p<0.05 
threshold (Bonferroni-corrected). Once again, there was a tendency of higher marks than those received by the 
amiable social style, though without reaching the threshold of significance (p=0.069). 

The dependency analysis of content scores with respect to the evaluator's social style showed a trend, though 
without achieving statistical significance (F=2.498, p=0.059). None of the post-hoc tests exceeded statistical 
significance. 

The analysis of content scores according to the evaluator's style showed no statistically significant difference. 
The analysis of appearance scores given by evaluators according to social style revealed a significant effect from 
the social style factor (F=4.312; p=0.005). Post-hoc analyses showed that the subjects with expressive social 
style gave higher marks than the analytical and amiable social styles, with a statistical significance which 
exceeds the threshold p<0.05, corrected by Bonferroni. 

3.2 Two-way ANOVA Analysis 

The 2x2 ANOVA variance analyses allow us to split the effects of social styles of evaluator and speaker, making 
the results independent from the distribution of social styles in the sample (see Figure 1). These analyses 
confirmed the significant effect that the social style of the evaluated subject has on the marks received for their 
presentation, both in content (F=3.35; p=0.019) and appearance (F=4.78; p=0.003). Analyzing the scores given 
according to the evaluator's social style revealed a statistically significant effect on appearance scores (F=5.43; 
p=0.001) but not content scores (F=1.69; p=0.167) given for the presentations. The analysis of the gender factor 
did not reveal significant effects in content or appearance, whether including the gender of the speaker or the 
evaluator in the model. Further analyses of the gender factor were discarded due to a lack of statistical 
significance. 

The post-hoc analysis of the factorial model including the social style of the speaker and the evaluator allowed 
us to analyze the social style effect, with independence of the evaluator-speaker style pairs. That is to say, the 
effect this factor has on each individual can be calculated, without being influenced by the distribution of other 
social styles in the sample. 



Figure 3. Results of the two-way ANOVA analysis. Split of the effect caused by the evaluators' and speakers' social styles (*: 
significant at p Bonferroni level <0.05; **: significant at p Bonferroni level <0.01; n Bonferroni = 120) 

In the case of the "appearance" evaluation, the analysis revealed that subjects with an expressive style gave and 
received higher marks than the other three social styles (see Figure 3). This difference is statistically significant 
for comparing marks given by amiable or analytical individuals, as well as marks received by analytical style 
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students. The expressive social style assigns significantly higher "appearance" marks than their analytical and 
amiable classmates. The expressive social style also received significantly higher "appearance" marks than the 
amiable social style. At a group level, no statistically significant differences were observed in the marks given or 
received for content between social styles, although there was a tendency for the expressive social style to give 
and receive higher marks. 

3.3 ANOVA Analysis Of Style Pairs 



Single-factor 

ANOVA 

Evaluator style 

Speaker style 

n 

Average 

Standard 

deviation 



Analytical 

Analytical 

10 

6.10 

1.45 



Analytical 

Driver 

24 

6.33 

2.44 



Analytical 

Expressive 

42 

6.64 

2.09 



Analytical 

Amiable 

18 

6.78 

1.96 



Driver 

Analytical 

23 

6.39 

1.50 



Driver 

Driver 

34 

6.47 

2.39 



Driver 

Expressive 

73 

7.36 

2.13 

Appearance 

mark 

F=2.464 

Driver 

Amiable 

30 

6.93 

2.24 

p=0.002 

Expressive 

Analytical 

46 

7.20 

1.91 

q 2 =0.049 

Expressive 

Driver 

84 

6.99 

1.93 



Expressive 

Expressive 

152 

7.40 

1.73 



Expressive 

Amiable 

67 

6.95 

1.68 



Amiable 

Analytical 

14 

5.57 

2.38 



Amiable 

Driver 

31 

6.65 

2.09 



Amiable 

Expressive 

56 

7.18 

1.81 



Amiable 

Amiable 

24 

5.96 

1.49 



Total 

728 

6.96 

1.96 



Analytical 

Analytical 

10 

6.90 

1.97 



Analytical 

Driver 

24 

6.25 

1.96 



Analytical 

Expressive 

42 

7.10 

1.88 



Analytical 

Amiable 

18 

6.89 

1.81 



Driver 

Analytical 

23 

6.78 

2.02 



Driver 

Driver 

34 

7.24 

2.00 



Driver 

Expressive 

73 

7.19 

1.98 


F=2.121 

Driver 

Amiable 

30 

7.08 

1.90 

Content mark 

p=0.005 

Expressive 

Analytical 

46 

6.48 

2.08 


n 2 =0.045 

Expressive 

Driver 

84 

7.00 

1.81 



Expressive 

Expressive 

152 

7.65 

1.67 



Expressive 

Amiable 

67 

7.10 

1.80 



Amiable 

Analytical 

14 

6.39 

1.80 



Amiable 

Driver 

31 

6.76 

1.93 



Amiable 

Expressive 

56 

7.11 

1.84 



Amiable 

Amiable 

24 

6.25 

1.75 



Total 

728 

7.08 

1.88 



Table 2. Analysis by social style pairs. Characteristics of the average marks for each combination of social styles. There is a 
notable variability in marks depending on social style. The * sign indicates a difference in averages with a significance of 

p<0.05, corrected following Bonferroni (n=120) 

The post-hoc analysis of style pair evaluations (that is, analyzing each possible combination of evaluator- 
speaker social styles separately) revealed a wider range of scores in each sub-group (see Table 2). In the 
assessment of presentation appearance, the lowest average marks were given in "amiable to analytical" 
evaluations (average ± standard dev. = 5.57 ± 2.38), while the highest marks were in "expressive to expressive" 
evaluations (7.40 ± 1.73). The comparison between these two is relevant (average increase of 1.83, SE=0.539), 

















































Jourrnal of Technology and Science Education - http://dx.doi.org/10.3926/jotse.50 


although its significance does not survive a strict Bonferroni correction (n=120; p=0.088). The rest of the 
comparisons between evaluator-speaker pair combinations neither do. 

In content assessments, there is no difference that exceeds the strict threshold of significance, although there is 
a remarkable trend for the expressive social style to receive the highest content scores. The lowest average 
marks are observed in "analytical to driver" combinations (average ± standard deviation= 6.25 ± 1.96) and 
amiable to amiable (with the same descriptives), while the highest marks resulted from "expressive to 
expressive" evaluations (7.65 ± 1.67). The difference in marks between the "expressive to analytical" pair and 
the "expressive to expressive" pair is significant at the p <0.05 level corrected using the Bonferroni method 
(n=120), with an average difference of 1.17 points (SE=0.31, p=0.023). 

4 DISCUSSION 

In the assessments of academic project presentation, a large part of the variance is not explained by the social 
style of the evaluated subject or the evaluator. However, the social style of the individuals who participate in an 
act of evaluation of this nature does show a statistically significant effect at a general level and exceeds strict 
statistical thresholds in certain evaluator-speaker social style pairs. 

The first set of analyses revealed that in our sample group there are significant differences in the marks given 
and received according to social style. The characteristics of our study group also revealed a non-homogeneous 
distribution of social styles. Almost half of the students were of the expressive type, while only 12% were of an 
analytical style. This disparity in group size is directly reflected in the types of evaluations given and received. 
With this distribution, any bias caused by the expressive style in the given or received evaluations will be 
magnified at the sample level. 

The second analysis (two-way variance analysis) allowed for an isolation of the social style effect, independent 
of the distribution of these styles within the sample. This analysis showed significant differences in appearance 
scores given and received by expressive style individuals, who gave and received higher marks. In this analysis, 
there were no significant differences in content scores between social styles. On the other hand, in the style 
pair analysis there is a significant difference in content scores (despite the severity of the Bonferroni threshold 
at 120 comparisons). This difference indicates a very specific bias in the rating of analytical students by 
expressive students, in comparison with the marks they give to each other. This fact, combined with the high 
number of expressive students in our sample group, explains a large part of the differences observed in the 
initial analysis (see Table 1). 

In the context of higher education, we perceive a double value in the application of the methods and results 
presented in this study. On the one hand, knowing the evaluation bias and its significance seems essential for 
designing fair strategies of peer-to-peer evaluation. With a simple analysis like the one carried out in this study, 
this bias shall be calculated and compensated wherever medium-size sample groups are available. On the other 
hand, there is remarkable educational value in the diffusion of results like these, in order to improve the 
students' abilities to design and monitor interpersonal relationships between the members of their academic 
and professional teams. 

This study presents relevant limitations. In the first place, the social style of each student was determined by 
means of a self-administered tool. By definition, the social style describes behavioral patterns as perceived by 
the individuals in their environment. Therefore, it is always preferable to obtain it through third party 
assessment tools. In this study we opted for a self-administered tool due to its simplicity and rapid 
administration. In second place, we have the size of the sample group. The exploratory nature of this study is 
limited by its small sample size. In various analyses, relevant tendencies to significant differences between 
subgroups were observed, though without reaching statistical significance. We hypothesize that a larger sample 
would increase the statistical power, reducing the standard error in the estimation of averages and, therefore, 
increasing the statistical significance of the results found in this study. 

The high prevalence of expressive social style students (44%), along with the fact that the best marks are issued 
by the "expressive to expressive" social style pair, caused that the expressive group was, by long, the best rated. 
This phenomenon is in line with the theory of "perceived similarity". 

5 CONCLUSIONS 

In the assessment of short academic project presentations, the social style of the evaluator and the speaker 
play a significant role in the scores given for content and appearance. Raising awareness among the students on 
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the sources of bias is recommendable for developing their evaluation skills. The evaluations based on 
presentation content suffered markedly lower influence from social styles than those based on appearance. 
Thus, content evaluation rubrics are once again shown to be key tools in guiding raters. 

The expressive social style has proved predominant in this sample, though this result cannot be inferred to the 
entire student population. If the expressive social style was predominant, it could be confirmed that in this 
socio-demographic context this is a potential source of bias to keep in mind when considering using peer 
reviews in the context of education. The expressive subgroup would systematically receive higher marks than 
other subgroups. On the other hand, no significant differences were observed between genders. 
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